An Annotated Corpus of Relational Strategies in Customer Service

We create and release the first publicly available commercial customer service corpus with annotated relational segments. Human-computer data from three live customer service Intelligent Virtual Agents (IVAs) in the domains of travel and telecommunications were collected, and reviewers marked all text that was deemed unnecessary to the determination of user intention. After merging the selections of multiple reviewers to create highlighted texts, a second round of annotation was done to determine the classes of language present in the highlighted sections such as the presence of Greetings, Backstory, Justification, Gratitude, Rants, or Emotions. This resulting corpus is a valuable resource for improving the quality and relational abilities of IVAs. As well as discussing the corpus itself, we compare the usage of such language in human-human interactions on TripAdvisor forums. We show that removal of this language from task-based inputs has a positive effect on IVA understanding by both an increase in confidence and improvement in responses, demonstrating the need for automated methods of its discovery. Under review in Language Resources and Evaluation Ian Beaver Next IT Corporation Spokane Valley, WA USA (509) 242-0767 E-mail: [email protected] Cynthia Freeman Next IT Corporation Spokane Valley, WA USA (509) 242-0767 E-mail: [email protected] Abdullah Mueen Department of Computer Science University of New Mexico, USA (505) 277-1914 E-mail: [email protected] ar X iv :1 70 8. 05 44 9v 1 [ cs .C L ] 1 7 A ug 2 01 7